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ABSTRACT 

This report on a project called Performance 
Assessment for Self-Suf f iciency (PASS) covers activities conducted in 
the first stage of expert system development: the acquisition of 
knowledge to t>e used as the foundation for the expert system 
prototype. The project is developing an information system for 
anticipating the service needs of young persons vith disabilities who 
are exiting from school; it involves having teachers provide 
information a£)out tha functional performance of students^ and using 
expert system technology to convert teachers' assessments into useful 
information that special education and adult services agencies can 
use to anticipate service needs and to plan services. The data 
collection process uses the PASS instrument ^ which otstains teacher 
ratings of students in the areas of daily living / personal and -^ocial 
development, employment, and educational performance. This report 
focuses on the methodology used to acquire and eml:>ed the requisite 
expertise of recognized topic experts in the knowledge base and 
decision rules of the PASS expert system. The report outlines how the 
Delphi panel technique was used to acquire expertise from 31 topic 
experts, how a list of client characteristics associated with service 
needs under each of 16 service categories was generated, and how the 
results were used to structure the knowledge base for the system. 
Attachments include the PASS instrument, service categories and 
definitions, and examples of Delphi survey instruments • (JDD) 
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The pmiA of tkis i»pcv U to diow bow ntetbods in the sodal scicaK^ akfed ta &e {Hocess of acquiring kiKmto^ 
frommultiiAseaqwIifiwtliepwpowofc^^ The project, twnwJ project 

PASS fw Sjofimtim ^u$smgia far S^-Si0ci^* is devetepiiig m mfofinrtwo systan for Mt kiprt m g the 
semce needs of yoaog pessooi with diiri>mtiee wto 

enraing a certificate, or raKdung Ae aoaimum a^ ftw services guaranteed tfaeu under the Individaals with 
DissinlitieB E<hicatioo Act (IDEA). 

Anticipating Serrkt Needs 

TTie tiiaH«i<M fttm secondary «d»calk» to a&iU 

of pmss with disriiiUties. One k«y to assuring a smooth tranatka is to provide resource aUocstoiv sad set^ 
pnmders with the inftmnatioo they need to prefect aati^ Ye< obCaining that information and nuking 

these piojeciioDS in a w»y Aat is uaeiul to service providers can be pro^ 

(•ducation agency staff who oofmally sabont such data are unfirmiisr with the purposes, ftasctions, and service 
digibility requk^Mnents of adult sMvice agencies. 

The above pt<*leoss in the collection of accuntfe and uaeM data on the aotidpa^ 

as weU as the need ?o bridge die gap between the whicatiofi and adult service sectors, are widely acknowledged. 
ResearehNS and know ledge engineers at the Anwican Institutes for Researdi (AIR) are attempting to bridge this 
infbmati<m gap by developing an innovative approach to anticipating and reporting the service needs of exiting 
students witib disabilities. TTus fproaOx hinges around two ct»npot»nts: (a) providing information about the 
iwKtional performance of students th^ teKh»s can provide with accuracy, confidtaice, and minisnal burden, and 
(b) using expert system kch»A>gy to conv«t teachers' aMessmeoto into 

and adult s«vices ageaciea at aU levels can use to anticipate service needs sad to plan swvices for young persons 
with disabilities. 

n»8e two c(»npoiMit8 wiU openle as foUows. First, teachers con^ete the PASS instrument, developed in an 
eariin study by A» with weU-known tianation enpats and state and local administrators and practitiooers in 
^ialedacatko sod aduH service, and tried out by teachers in a anaUpik^ ThePASS 
instrument obtains teacher ratings of students for a broad an»y of fi^^ 

domains: Daily Uving. P«sc«al and Sscial Development, Employmeot, and Educatiooal Performance. TTie 
specifk skills and behaviors targeted on &e PASS in^rimient are ones that ar^ 

that have service implications. For exsn^le. very low performance rrtings on sevfiwl spenfic indicators - 
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M 'movw lelf aboot ia imamlnte netghlioriKxxl (e.g.. waUdng, tncycUng),' 'uMf pidHtc tni»pof1ition if avnlaUe 
(e.g., baa, twi)," "xmm taipt mi bos sdiedules when tpptopnae,' ete. — suggert diffiwing needs for sssi^aiice 
withnwbilityMdtim^oiWkmaspectt The PASS iastiuniBot abo piwides infcmn^^ 

studcRrt's tnuaing , eAwtfioB, nd ^ii^ymatt. » wdl « n»ior prc^em beliivkaB. No qiedel « w«MniflBt is 
nquind: ♦'ichesi con^Jtete the PASS bMod oo wh«t they ilieedy know about ttie student from direct ob«»v«tion 
Of odwr letiable iapiA. (See ^HoofAvncitf .4 for a copy of the PASS iaatnuneot.) 

Tte second compooeat of thu new apfHOKh will be to uae expert system tedmology for projecting 16 cat^ocks 
of servicacatia^ for indivkfaiala and grwipg baaed on <lita from the ^ (See Amcfcweitf « for 

the list of Ae 16 swvicecatefories and their definittoos.) Bq>ert system leclnirfogyrrfiM to AatbiwH* of artifia^ 
intiaiigmce dedicattd to leplicating the dedsioo process of ree^^ 

(i.e„ coat eflfoctive) and i^i^.' Bxp«t systems have been applied sacces^y to a onmber of proMems in 
educati<» and have jec«lly fo«nd applki^on in meeting the needs aasodated with special educ^tiou.' 

y 

Hie purpose of the PASS expert systwn is to convert tead»en* ratings of students on the assessment instnuDenl, 
c^ by case, ii^ imdtvifhiai or aggi^pted pn^ectioas of adult service aseda. The syst^ is not mesat to provide 
a disgnoais of any one stndMt's needs itf the local levd (although 

be for training, conyanson, and initial ccnsultatiott), lal^ 

for ai^t service providen st aU levels 0«5al, state, and national). Becsuae of the daU processing emphasis 
associated with tiiis type of system development, &e PASS exp«t system is hmng desigtajd to con^emenl state data 



s3rstaBs. 



Expert input U particalariy important in development of ^ expert jys4em and is the key to providing a reliAle and 
usefol data system for i^annera, poUcy makers, and piactitioaere at aU levda. The si*ject of this paper is the 
metiKJdotogy we used to acquii* and embed the foquiaite expertise 
and deci«oa nilea of the PASS expert sy^^n. 
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BaiT, A. & Feigeohanm, E. A. (Eds.). (1982). The hcmdbct^ t^ait^dai infeUigence. Palo Alto: H«iisTech 
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^ See, for example, iJasso^bring. T. S. (1986). Specialiad servwes and expert ^Pff^J^^^'^^^^^^ 
the potential snd waUty. Computers in the schools, 3, 173-183; snd Hofmeisl», A. M. & Lubke. M. M. (1988). 
Expert Systems: In^c«S4«»» for the diagnosis snd treatment of learning disabilities. ]mnm ^fmmQmmy , 

11, 287-291. 



PerspictiTe 



Rqjrnenting Knowiedge B«e DiTersity 



In ^oenl tern, knowledge acqoisitiai «nd the eccoupBiyiiig pioceea of stncAiniig ^ knowledge invoivee • 
jHt)c««inwli«AlMmmoxp«tMew«t«^aodt^^ TTie 8lnictii«d 

knowledge bMe, whkA coossii of fiK^ «id deciska 
ofthesyrton- As critics of ertifkaid atfelligeoce 

eqwrtsysiOT is only 18 good «8&e«iptt*befhnn which By incoiponling the knowledge 

Hid heuristioi of » hiuwBi infonnetion proceeww itf knowtodge be«>, any eipert system at best reflects die 
idiosyncnacs, bisaes, and fiainea of irfweoce of the expert fitom whidi tt was imxtocod. 

To meet the demands of a national data refKMting todnwlogy, we 

the PASS exiwt system is buUt naist be ciedil^ to a bio«l spectnmi of 

ftomnnUtipJen^ionsajoundtheUnitedStatBa. Whereas nmy expert systems idy upon knowlet^acqoifwlsokly 
fhm a sfavks expert, our spproadi has been to deviito from that moddl. building the PASS informalioa system 
to reflect knowtedge as U is distrflwted tbrw^hoat the fi^ TTw. we sou^ to obtain moltiple i^ from a 
diveise and repree«t^ve gto^ of experts reflecting the ftdl ^jectnim of disalality c^egories and adult seivice 
areas. 

Sevenl techniques in the sociaj sciwcea have been developed over time fiw acquiring wtpwtise fnwi multiple 

These techniques vary in (a) whed»er they are itentfive in nature or are con&icte^ 
(b) whether they aUow for ♦ke^fccediscusMon or take a noniitoactiveap^^ Itentive procedures include 
mechaniiuns for building consensus over cycle* but tend to be diffkuU fo coordinate and incur the -A that 
p«ticip«its may drop out beftse the cycle* have concluded. In coitfn»t, sin^e-round procedures are quicker to 
oooidmate but are not 8df<or«ctii^ and nwat obtain aU uaeM informatkm 
With respect to interective vs. nDDinterective 

real time pfoviding a synergMm for producing qualitative input as participants respond to otiien.' probings and 
feedback. Noninl«ctivetodmBjtteelacktiiesyn«g«moffcce^fK»di8cuso^ 



' cf. Shneiderman, M. V. (1982). Itentive procedures for forming expert judgmeots. .'*«'««^'^^5 
Control, a trenslation of Ayianuika i Tefem^duuUka, a pubUcation of die Academy of Sciences of die USSR, 43. 
4, 568-572. 
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to coodact - ii p«tki|Mti do oot oeri to timv^ 



Tl» fod of taKWtodge acquMitk* in Ito im:^ 
oMg 1 ooniirtenctiva, itenlive s*!^ 

(kci8M>on»k«ii8iiig»&»4o-fiK«,iK«it«^ ToiccoiM«»di«tgod,wb^wth*iiiodifi^^ 
of llie Ddpfti imw«, Ml ttentiw 

cMpotam to «ll»ri«iw ofsniaa, wfaclMice to wvw • pi^y v«c«d ofanion, tad odwr iablritwy pioc««». 
With tbe Deli*i jwjd » • stMtiiii ixwtf, 
expert, to syitfhmDrfi*^ 
fiiff dto tt^Mt flystnn. 

The Ddpfai Pfead 

Origifldly coiic«ved over thirty ycOT «go, the Ddphi 

Strt««od«b«)id*.iiidh»be«»cogm2rf«avtluri>tet^ AIR h« found the 

to^Bkiw to be perticuUity iiwfW «t««^ 

diaririlitie*.' TT»Ddphite(*iiiqi»cti«iitioiidlyii^ 

gfDi9 of experts. la the fii« roiiiid, expert! m« edad to geoento in^^ 

TTietiafbnMtioiiiitheosuiijmM^ The ultimateimipoee of Delphi is to derive 

cooseBaw ftom muitipie somcee of input It AouW be meatioiied thet altfaoDgfa Delphi p«i^ m frequently 
qusntitative in the Bumnaries ciiculeted to pMtidpetiiig experts, rwisiooists of tt» proceed have reMoaed that 
too nBich of a quantitative ea^hasia can lead to artificial coovefijeace and suggest modifying the technique to 
piovide qualitative fee^Mck instead.' 
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* For an exaavle of how the dynaanca of fi«>4o-fiKe diwaission can serve to suppress idess lod polariie 
decisioa pfocesae^ see Jaaia, B. (1971) Chtjupthink. Psydtclogy Today, 5 (6), 43-46. 

«Lawtoiie, H. mA Twoff. M. (e<b.) (1975). The Delphi Mahcd: Techniques and AppUcatUms. Reading. MA: 
Addison Wesley. 

•Boo». John H. (1986). Expertise trtm^ for expert deii^n (Vol 3. in Advances in HumKi 

Factofs/Ef^HKMnics). Amsterdam* The Netherlands; Elsevier. 

^eisg«b«, R. A- and Snnth, C. A. (1977) In^mmng vocational education services for haadicai^ studento. 
(Report #0007500391). Wadiington, DC: Bureau of Occupational and Adult Edocatjou. 

• cf. Sdmeidwinan, M. V. (19S2) op. cU. 



W« used the bail fe^nwfrft^BXMtel: itenlive cycles, ispiit fipom nsilt^te estperts and balnoed coastHiMndet, 
«iid«fbcy»oodrt»deeinBdbycoi»«i«8tobe!nostci^^ Tlie Ddphi v« coDdwtol over tlaw ite^^ 
touKb, described below, between Febrauy tod May 1991. 

DKteSomn. We lelected 31 speoai educatioo eod acbilt senm experts to pwtirapeie in die De^ pitwew. 
Tliey wefe ftom diveise geogiii*Mal 

»peclnimofejtatesenrkesi«M,iDdtnwMi«w«iti«^ B«* «qwt b«l at leest three to «Dur yeen 

of direct coBt«* with dkots with diaabUitiee aad » 

the need for easH^ services. {AU 31 experts pwticip^ in Ihe fast two iwmda of die D^. bitf oi^ 
was unable to napond to Round 3.) 

Romd 1 proccdi^ We aAed the experts to idertify cliert cha«cteristk» ihrt for ewA of 16 service cntego^ 
ww«Ukdy to trigger an eaaential service need. If an expert did not foel qualify to respond for a paiticolaraBrvice 
category, they were asiwd to ma* 'don't know- on thai form. Experts were alao invited to comnwtf freely about 
dieir decisioos snd about die definitions cf service categwiee provided. Examples of the Roimd 1 fermst sad 
f«8ponsee for severel service cat^ories appear in Astadtmmt C. 

Experts had nev« ae« nor did diey have success to die PASS insmmiBnt while con^^^ 
round of Delphi); dierefbre. dKar re^tonses wae based eitfiidy o^ 

by client ch*rec«i»isticsspecificaUy addressed in d» The 3 1 experts who con,^ Round 1 provided data 

of outstsBdingiiuality.depdi, and variety. This ridi and comfaeheosve data base fonned d» founded 
for 5ub»quent rounds of Delphi and for the devek^wneot of the PASS exp«at system. 

AaalyssoflUmadiaatn. Four Am ^ect staff conducted an intensive, twonrtage analysis and rrfinem^ 
Roundld^ ll»fU8t step was a detailed Hiftlysis of die data and daasificat^ 

die nnge of initial responseo. Tlua fii^ ^ was conducted iiid(!|>codently by two ^ Thesecond 
step was the sy«thew aoi «fii«««t 

TheaaalyasedressitowereorganiiBdasssqjaratesuninariesfors^ Each summary. 

Of oudias. i»«fflted dtt chaiacteristk* diat had been idea^ 

essential safvice ased. In lAiitioifi. ^ific exanyles of tisese cUent characterise as encountt^ 

werepmvidfid. TTutsimunaiy foimcdtbtsubst«K» of the materia 

below. 
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Rouad 2 nucedat. Tte pmmfy pnpooe of Rouad 2 wm to tdeotify tfaiMe dieiit cl^^ 
ioduded in Ae decuioo rate for tfw fiatpett sysbm (i.e., die "if-tiMo' rate stro^ure ^ fte syitaa softwsie tuee 
to inoje^ eervice necdi). B«c«^ expert system must tkfioe a fini£e and ec<»03nic»l set of nilet, oech Xpert's 
tMfc ia Round 2 wts to idei^ <mfy teee cUeot dtttacteiistics in eadi of the 1«S wmmtnee (the o«t«^ fiom 
Rottttl 1) tto would definitely (xiggn tlmr dedsion to indkite a seivice as Mag essential for the odividuaL 
Hieiefon, for Ramd 2, we arfad e«i»rti to aaaeai the cUent cha«*erial^ 

invoftaace for being included in tl^ expert sjrsteni, umg a forced-cbotce ranking system of V«ry baponam, 
Uttiit^tmnt, or UHdedded. The rating sdwnB con^dled the experts to identify ani^ 
chajacteristica ftom Round 1 dwse that were absolutely crit^ 
system. 

Secondary goals of Round 2 wwe to vfeifywhelhw the 16 ammariee (outlines) accurrtely airi adequtfely captaied 
the mfonnation provided by experts during Round 1, to identify any gaps or inappropriate daasificatiooa in dw 
outlines, and to assess the adequacy of the service ]ist definitioiw. An nuunple of the Round 2 fonoat fiv one of 
the 16 service ctf^mies iqipean in Atusdmaa C. 

Aailysb of Round 2 ^itn. In onkr to quairtily Round 2 readts, responsea w»e coded on a dueoiwint scale: 
Important - +i; UnimportoM - -1; UndedtUd - 0. Items widi no fBspooaes wb» coded widi an asterisk (•). 
Sum values *w caksdsted for ee«* clieitf chanKteristic rated and 

who ntiad die chartctedstic to obtain a proportional eating, or 'tsting ind«- (i.e., die sum value was divided by 
die number of feqwodents to obtain an adjusted sum). We establidsed a du«diold value for die rating index of .60 
and above to id«tify dMise chancteristics diat expert cwisenais indi 

base and die nde strwluie for die PASS expert systwn. Characteristics tbat were at or above die dueahoW value 
became dw basis for ahortnwl lists of cUeot characteristics, one list for each of die 16 seivke categoriea. 

Round 3 procedure. The priaMiy pmpoee of Round 3 was to 'map' dioee client characteri«ica diat experts 
ojiiffldeied a»®irifl^ for inctaeioo in d» e^ 

ictentifygapa in die iMtnaneofs coverage. Tljis 'map" was to s«ve as a guide, or ten^Ute, for die committee of 
experts meeting sG^boeques^y to use in fornmlating preliminary deciaon rules for die exp«t systan. 

We pnmded die pamd widi a cootesat outline of die PASS and a lirt of d» 8^ 
e«:hofdwl6 9ervj<» c^egoria (a content outline and exanyle of one service cate^ appear m 
We asked diem tr> iiMlicaSe one or more t<^c areas in die PASS'S outline widi 

lists might be aast«i»ted. To simpUfy die experts' task, we cuaducted owo -manMag" »oalym and suggested 
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dmragiecmeitf w^ourjiKl^iMtfs. Alio it this tim, ioviSed commei^ draitf tte gene^ 
instnuBMit. the 16 of cJkot dwBCteristia firom Round 2, and the seivtGs catefofy defmitioas. 

Aitaiyk of Royod 3 In gcoeni, exp^ inpi^ from Roraid 3 indk^ coBstdeiakle agreement witb die 

pralimin»y napfw^ lecomawidaftfliw We snmmariaed expert nqtnt from Ronod 3 and created a matxix to 
deacribe ttie ovenOl tiafcage of PASS domaina to aetvioe categoriea reftoc^ in Round 3. Thia matrix (aee 
Attachmem 0) provided a fiwpevwwk ft» the delihetaiiona of tiie Temptee Crannatiee in ftwnBilaiing preliaBaary 
deciaioo ndea fior dw expert aystem. 

Vsbm fl» Dciphi Rcsidts to Stmcture the Knowledge Bmc for the PASS Expert Sjstan 

After ccHopleting aU tluee rounds of Delphi - and aa an int^ p«t of obtaining expert and end^ input 
ttuoo^iout tbe pwgect - Am convened a three-day meeti^ 

TTiieeotttBdecoMultwrtaandthe Ampf»9ectteamparticipetodindieT«n|rfa^ Twoofihe 
conautamta had p«rti«pel»d in the Delphi iMOceaa. One vwm a vocational rehAilitaiion counador for the State 
CaliforaU Depertmeitt of Rchabditatioe; hia perspective rep^^ 

withdisabititiea. A second outsde consultant was the exetaitive directs of tt aAdt independert Uvu^ 
pei^»ctivereprt«ailodaservice provider ftw'low-fi^ The third outside c«isultant 

was a nationaUy known and reapected univerBty-based txanstticn reBearchw with expertise in assessment for 
tnnsitioo plasoing. scales of independent heiiavior. and the role of interapocy coUti»ration in statewide trensition 
{banning. 
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Tonplate Committee Frocee<Sngs 

T1»e meeting fbltowed an agwdath^ involved two principal acti^ (a) . prx«a«rf of expert deciaon 

making stntegsee wng three "case study" PASS instnwwtfs, «id (b) die development of preliminary 
deci^m rolea. 

Protocol annlya^ Protocol malyss i» a knowtod^ acquisitioo tool borrowed firom cognitive psychology diat 
dlows expert sy»t«n deaignen to study tiw cognitive decistoo making str^eyies of experts. Quite ain^rfy, die 
technique involves giving experts a task to do, such aa saving a problem, 

mental activities they undertake to coB^l^e die task. In die prtsaeat context, memben of die T«M«to Committee 
w«e asked to considw diree s&mulated case stuues of exiting students widi disabilities. The case studies were 
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ForthefiMOMe, «Kfeiii8ttiwoffliBComiiiitteew«8giv« Thethre© 
extenid cooaallMtfi wtw A« Miwd to identify whi^ 

wouW recomnieod b«wl ooly oo the ditfi coo^^ Hie three «p«tt were tatod to trite tanw 

vetbeUnngonay the BiMtfripra»MW they were F« the eecood end thin! 

ca»e hi8tofi«^ expem wwe «Aed to leech oo«d^^ 

they used to wech Aeir dedakoft. Wb« fiaiAed. they wwe iavited to ihw their ded^ 

with the gfwip end toe* probing ipwtioo* of od^ By the eod of the exerciee, the COTmattee bed 

beeo ible to ea eDcoangiBg level of Mnwaitts on all Aree ceee histonee. 

From die protocol exerciae, the Am tewn obeenred Uiei die e«pert dec^ 

wwridngdirough die PASS infbnBrtioBhien^^ Ai die espati leported, stK»U wee too difficult to meialeia 
•U iflfimnetiooel iteaii of die PASS in manoiy , diey 

isdieyTOt(«proceMi^nwlto«»diHnkingbycogmtivep6ydi^^ For enMnpte, in deckling which eeivice 
cit^oriei wt« epplkabto beeed on PASS dett, they woidd go to e«A of d» 

of ^fASS ^Mfy LMng, Pemmal atid Sodal Employmmt. EdMemUmai Petfimnaitce) md 

derive from die specific bdiwiof ntinge, ove«U sid^^ 
ineed&eiee. Then, witii overaU sdgective eooiee in mind, diey would 
«ws ageinat eaA odier end egeiMt infbfmrtion fwm 

cooclusioo. The importance of chwUdng bec«ne obvioua when extracting deciaioo nilea. aa will be 

described preseody. 

Before moving on to die ^wcifkatioo of decision ndea, die Ten^ 

PASS apm system shouklprodMcetaA how $ht systems^ -fteaHmmttee 
concluded dMtf die PASS EjqxrtSy^ahonU produce two typea of coodii^ Fiisl, die system must c«icl«de 
which of die 16 aeivicaa. eidier singulariy oc in combination, wouW be need^ 
«rtei»ddieauspiceaofaAilt8e«vic«. Second, die syatan should also provide informatioo forjudging die 
ofdiatneed. fa both ceaee, die ayitem should alert die end-usi» when aoJ enou^ infonnation was 
provide a tmatwoiAycoattaaion. Thtia, in addition to indicting whk* of &e semce cati^oriea a» needed^ 
systHn shouW indkato whediec that nB«d WW considered to be of 
immediate at««ti on in«^«ctive of budgetary and od« 
pnwider may utiU» a greater d^ of peraooal discn«k»). or 
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tflf pitUwydecWoandcs. To dev^ a pftdiiiiis«y lec of ipedfk deeiakn 
CoomiltM fbOaweA • proem ftit wis • comirf^ fwwwl fm tbtt used (toing protocol osiyns. Wbswi 
protocol iMlyiu nquind caqieito to bega with a o(»ii^led PASS iaatntnieBt and to move toward coodttnooi 
wgwlmg a«vk» cateiofw. nda qw^^ 

•U the neccMuy proooaditiaiM in the PASS tkat would be uadU (or mlm^ nisable iervi«e predictioas. Hui 
procw mpBitd that the «q« wwk hadwait^ 

(i.0,, the ol^ecta) that wooW be needed to leadi a coacfaMoe for each invKe c^egoiy, (b) idattify leveU of 
severity among thoee item Aatwoold lead to datiag^^ 
(c) decide how inforairtioo ftoo the PASS iteoii wo«W b^ 
dedde what degiee of nri«Bg iafonatfioo w«iW th«^ 
coDcluskxi that saofv infofiiittioQ waa aeoded. 

To aid in diis hieiaichical biowledge aapuAtiott, a hypertext anapiitemed diM«y tool wa» conrtraclwl using 
ToolBook' for the IBM PC. The toed, diii^yed on nadtipto video dis|rfay tenninals so mM*^ 
cottU follow, pflfflvted oienbeni to oeato a^ 

finer levds of detail. It did this by preseoting an initial scieeo conposed of the more genra^ 
Disabling Condition, Daay Uviag adlls. Social Devd^^ 

Behavion, and odieis - see Figure l) and lh« allowing &e Coauiattee to select any single calegcwy fof fu^ 
expenssoo and definitioo. Since the tool had beso written in a hypertext language, it allowed for fiwt sequel 
to finer levds of detail by simply perilling to the de««dcat^«y^^ The screen wouW 

lefiesh itself and display prompts necessary for consideration at the lowr level. Using this technique, the 
Committee was successful idsBtiiying diose preconditions ihst w<»W 

service ca^«ias- 

Roidls of the KiwwMgc AofuisitkMi Rrocass 

Having extncted a eonvkle set of decision rvks from domain experts, AIR's knowledge engineers have now 
convleted an initial prototype of the ocfflv«^ri»iknowkdgeba»^ This first prototype has been developed using 
Nexpert Object, a tiW!»rtahle and interoperaSriepropanmiing 
prototype win cotfinBe AroBgh snoceesive revirioM is it is 

nKl in the field. A thoroi^ valklatkn of flie knowledge base, in terins of pen^v«^ 
od>er expert samples, remains outride the scope of die preeesrf coop«^ 

for future work. 
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« Wtei MMitel Jw* iiiBWl III 


1 


C3MlMtMniiGtMt4*waitey || 



hgatl. Baua^i^ of MBCM from an JJR tool ^eugpta to for ow of 16 

service cil^ories. 

AdditioDeUy. AIR stiiff hM develo|«d • rev««l ^ 
b«ed M iapitf fttmi Ddphi, T«npU»e, «id <Hte 

an AXR-snidy that p«N»ded die study diiw^ Folkming • iwkw by the sponsur, the ii^^ 
c<«verted to the iMchiiie4«ii^ The knowW^ •apiisitioii 

stage dso p»ov«led iBfornrtkm ft^ 

A$uidimatt B), that are used is outcomes of expert system proceaang. 



TTus report co™ ac<ivit» coodi«t«ri in the &^ ^ 
«:qui«tioo of kiwiiWr to b» "ed « Whereas expert system 

p«goct8 in asmy od« «tem«tB8 are coostiM^ 
coostnicted with ii,!^ fttm multipto experts, fa^ 

Because the pnqect must incorpoiate mp^ from multiple soureee. it is utilizing methodrfogiea in knowledge 
•cquisttion that are designed to buiU conseosus iteretively from a div«sity of inputs. 
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Knowkdge aoqnMtoM fot tbe pageet begn with a Delphi technkpie used to extnct oobmbiw ftom • pmd of 3 1 
«aq»fiiw5lh«x|WT«owin<iBtBniiii^ CooduciiBdby laiU 

<wer tbiw wsceiwe iwaid^ the Dc^ii to^^ 
aeivk* needs umte ««* of 16 wvice c«t^^ 

to be eepecidBy evocMive of 1 jsfinwy Med for aerv^ 

syrtem'i kiiowl«^ be«. lal (c) provided « iait^ 

thePASS. The Delphi tecliiikpiew«sfelk>%«d by eOBe*^ Thepnipoee 
of fl>e Twi^leto ConoBtu* v« (») to ptovkle ki^^ 
jmKocol, of how e«p«t dedBoo iMto idBtfilied eA^f len^ 

PASS, (b) to Hie seiiilts of the De^ piocew m « «* *rf mta for the PASS expert 

syitom, and (c) to a»«der end feanmiMcid twmoot to the PASS iastfiiiiieiit end service lisL 



As this ph.se of knowledge loiwsitioo eods. Pi^ 
iaftHmstioii t^wtling the msBoer ia whi^ 

for mlicipetingedBit service Deeds itfMltond,s^ Efforts h»« be« tmdeiwmy to leprodoee 

iMthMytliMnMr pod of inforiiistk» into sa effective 
system of iteretive prototype developiiMt, ho«w«^ 

scquisitioii. As die evolving knowledge bese u tested sgetmtf new PASS date (from a nsttonsl fieW t«t). the 
knowledge base will continue to grow snd be r^uwd. 
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CONFIDEiKITIAL INFORMATION 



Perfbrmanca Asseswnmt for Setf-Sufffckmcy (PASS) 



II 



OmOMAL 



0OOOCDKD0OO0 

<5)0000<3)©®00 

!®000€ 0©©0® 
©©000,D0000 

©©©00^®©©© 

©0000I00000 
©©©©©I©©®©© 
CD (D ^ ^ (DKD CD 0 ^ (f* 



STUOCNTS 




STlMXMrS 



RATER'S NAMS* 



0 Mail ® Nmalt 



0 
© 
© 
© 



STUOiMrS 

n>#wcrrY 

0 SiKk 



00:^010 

■ — ©( 

<2>0®|0® 
©I ©|©0 0 VVhit» 

©©m © AsafVPicifici^andsri 
000! 0 Anwican Iwlian/ 



Als^can Nativ« 



RATER'S PHONfi NUM8CR 

1 



SCHOOL NAME 



OiSTRKrr MAME 



Vm mm m *• <Mel««r ar ottut 



SCHOOL 

ZIP coot 



®©®®®l 
O0GOO 
0000© 
000(1) d) 

00000 
00000 

'00000 
00000 
©©000 



SmOEtfTS PESiOMATED D«A8UW0 COiflHnOli (Rfl«ct» Hdtnl fpofting ctagori— • Mw» oim) 



0 MantriR«>rclltx^(M«fc$Mrtyif Mfl) 

0 Mild 0 iVIodtratt © S«v«ra 
© Sp0<ch or Languagt impaffmant 
© Visuai Irr^WMtmnf or BMftdfiMi 
0 Specific LMfnin^ Qgabititv 



0 Sorma EfTTOtioruM OistwbafK:* 
0 Orthopedic Imptirrnant 
0 Other H^th bT^*eiriiient 
® Hwino tfroeirmem or PMfrww 



0 0«sf-Biindrw9S 
0 Autam 

0 Traumatic Brain Injury 
@ ^%4^tp^e Daabitttws 



• WhM win/dkt atudent ieav* school7 



• Under what eondttton wM/did atudwit 
leeve aehoo47 CM«ii one) 



0 Grackjata with d^Homa 
0 Graduatt with certficatt 
0 Reach maxtmum age 
0 Dropout 

® Other (Spadfy:) 



• How ioftg have you i»K»wn thia etudent? 

©0-2 m««hs 

0 3 - enrwitha 

0 7 - 12fTWtha 

0 Mor« than 1 2 months 



L 



O0 
0 
© 
0 
© 
© 
© 
© 



0 
© 
© 
© 
© 
© 
0 
© 



01® ®< 



• In whet kind o9 eiaaiMVwm do/did you heva contact 
with thia atudem? (Merit one) 



0 %guiw dMs 

0 RMO^tfiM room ipufl out) 

0 Sefwate^M 

0 Separata school 

0 Vocaoonri comnrHintty-based class 



SigMrtiM of Retar 



Date 



• Rater'a RelationaWp to thie Student 

0 TMCher (provides ongoing, direa if^struc^) 
© f^gi^ educetnn © Specisi education 
0 Coniutting »acher (prevKiei speciai iducation support) 
0 Abrfc expenen^/worfc study coordinator 
0 Othar (Specify:} 



BooMioo«oo«»ocooooooooo ^ 6457 

^ tnjAse DO WOT white m this area on use this siwal wummw 
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Mart «tf scttmgifl) #i 
wftcft p<f fo r m an c» * ^ 
known n voo- (O****^ 
on p. 4 ol fciflSucMnij 



8 - 

W - 

H • 

O - 



OttMT 



f%f f d n^nc§ fl#tinQ CocJw 
iOcftntd on 4 ^ kistnictions) 

OL UMMilBrM 

1, Om fiai or MiHM do 

L Oooi or cow tfo loitf i 

3. Oooo or Mn do wMi t^;^ 
mi li t i nB o or wp« ir ia i on 

4. Pootorcofido imt i M i iti t w tfy 



Functional 

Performance 

Indicator 



S W H O 

0©©® 



0 12 3 4 

00000 
00000 
00000 
" 00000 
00000 
00000 
00000 
00000 



a Pmonal and Social D«v^opfnent 



B-1. Communicatkm 4iaHt 

a. Pays •wnoon arttnowtodgts stJiro*) 

b. R««»fidi to 3om« form of languafl* {•■g.. oral, wnmn, *gns, Braille) 

c. ComnrrunicatM banc rMMds 

d. SpMkstntaHi^ 

e. Tates part in rrtavant convtrsatwos 

f Exi^sssas personal faalings approprtata to tha srtuatwo («.g.- f«ar. joy) 

g. Asks quastMHU ara ra^xxids appropriate 

h. UsM o rta phon a for information or wswtanca whan naedad 

). Obtains information from want ads or MJajrfww diracttirv 



S W H O 

0®®® 



0 12 3 4 

,"00000 

00000 
j 00000 
• 00000 

00000 
■ 00000 



B-2. Acting r«sponaibly 

a. FoNowsrx^ 

b. Admits m«iakaa. off«» to ractifv of judgmant or conAiCt 

c. Structurw tima wrt {• g. mamtama a scftadula) 

d. Assota othars wt^ naadad 

a. Makas dacisttiw that ara raasonable and tirnalv 

f. Follows tfirouflh on plans, decisions 



s w H 0 

0S00 



0 1 



00000 
00000 
_ 0000© 
00000 
00000 
0000© 
1O00© 



2 3 4 B-3. Coping akitis 



a. N/teMges own emotions (a g. frustration, angar) 

b. Rasim negative pe«r pressure 

c. Responds a p propna tety to (Khars' nagetive remarks 

d. vvbrte out con^)rom»es 

e. Adai«i to ch«igas tn situawro (e-g, new activitias or pe<«)la) 

f. Functions wrt under work or time pressure 

g Wenlifias and takes steps to deal with interpersonal pnAJems 



S W H 0 

0®00 



0 12 3 4 

"^0000 
00000 
, 00000 
; 0000® 
1 00000 
I 0O00® 
' 0000® 
• 00000 



B-4. R«tationfthip« with oth«ra 

a. Greets people wop"«^ 

b. Shar«s appropriately <e.g.. property, time, space) 

c. flespects nghta of others {a.g. the^ propprty. privacy) 

d. interects appropnately vwth in<avidu^, inciwfcng strangers 

a. Inwrects ^(^c^taly with people in wrthonty 

f. Interacts appropnately as a member of a group 

g Makes and maintains ctose friends 

h, Takes part m le«ure-tima social activitias or organaed sports 
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B BEST COPY AVAILABLE 



Matt ^ smmgna tn 






(mown ID you. (OaAntd 




Mmcaent) 


S • Scti 


ool 






M « Hm 




0 - OtfMT 







S W H 0 



^ifformanct Rating Codw 
iO«finid on p. 4 of Instructions) 




Functional 

Performance 

Indicators 



0 13 3 4 

■^00(3)0 
00000 
00000 

" ©0000 
00000 
00000 



D. EcHicational Ptrformanca 

RM^no (ifuMM pictim. ^aMringt, BmiNs, or oth«r t^^ 

a. Raadt and co m pwN n d g survivt T—^)ms wordt {a-g., stop, danqtr) 

b. RMdsandfoUovMSMiipto 1-stapor 2-stapMittrucfiona 

c. Raads ai4 fotewt nndtseap »mrvctio>w 

d. R— ^ and ccw ipiaha nd t tha n»w«p«paf of »ira<af panodicab 
a. Usas rafaranca maiari^ (a.g.. (ttetionary, ancyckspadia) 

f. Raads and cwnprahands taxitxMks at grada iavai 



S W M O 

0®®® 



0 1 



3 4 



;"^O®00 

i 00000 
1 ©O000 
00000 
I 00000 
I 0000® 



D-2. WHting (t»v h«nd. typ«wrtt«f, computer, or othar Mairtiv* davkw ^ 



a. NAfritat OMm rwrna 

t». ProAicas iagibia wri^ 

c. FiUf out fmipla forn» 

d. Cofnpoaat simpla sanfancas and sffnpla paragrapha 
a. Compoaas tattan 

f . WHtae compositions that convay idan ciaarty 



s w H o 



0 12 3 4 

""©OOO© 
, 00000 

! 00000 

0®®® I ©0000 
' 0000® 
I 00000 
' ©000® 



0-3. M«h«n«tic« (m«v Inducto u«« of M»4ftiv« dovicoi) 

a. Counts to twanty 

b. Tatetmna 

d. MMiuaav parfomis basic addition and sutmacticn 

a. ManiMlv parforms IMM miMpNcatm and divwon 

f. Solvaa »iipia word proWanw 

g. ManuaOy calculatas fractions and dactm^ vaiuai 



S W M O 

0®®® 



0 12 3 4 IM 

■^000® 

®000® 
0000® 
®0000 
®O000 



Aeactomlc work habto and akitia 

a. Rjflows orM or tntKprativa Isigpi ianguaga) instructior« 

b. FoHowvs writtsn tf^ructnna 
c ComplatBS Msigrmants on «na 
d Compiatas Msignmants sati^actority 
a. Tineas notas in data 
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F. Qthf Infomiatkin oii Emptoyment 

1. TlT4t T*"**""^ tYff^^«»v wQffca in^lnwHiwittv on iOb-Ww — ionniiitt or t— M for {M^ on*} 

0Oniin. (D iSmm. (2) 30mtfi. 0 60 iron. ® 120 mm. 0 Don t know 



(Mttfk M t»i«t apply) 

0 Othar (Sp^afv:)-^ 
0 Nona 

0 Don't knoMT 



TMa itudam's voeMigfia} p iati aia i ton mehictad: 
0 Pravocaaonal HtMa tr^nrng. caraar awafanass 
0 vocational aducaiian. ganarat 
0 Vbcationai aducatlon. job-ipacific 
0 Schooi-bttad woffc wf mm M iM 

Haa thia studant axptaaaad a daaiia » pufaua tha fotto«(i*i« araaa attar iMvlng tcHool? (Matfc aMtfiat apply) 
0 wbrk 0 indapandam livinfl 0 Othar (Spacifyt)- 

0 Poat»«»ndarv aducatoo 0 MtHtarv ® '^ooa 

0 Vtocatwnal training 0 Homamakar 0 Don't know 



Q. Othar Informatiofi on Traifiing ■«! Ediicatioft 

1. whtehoftfiafolte^wingtvpaaof QSQwcatiooaltrai^ 
(Marie aU tfiat apply) 

O Tfanapoftatwn 0 Shopprng 0 ^ 

0 ResxtenttAl 0 Accasaing agancy servicas W Don t know 



Thte •ttMteni't moat racan^ 

0 Gradt 2 or towar 
0 Grada 3 • 6 



raading iavai ia: (Mark 

0 Grada 7 - 9 

0 Gradi lOorNghar 



i) 



0 No tast n»ats 
0 Oon't know 



0 No taat rMidts 

0 Don't krww 



Thlaatudaot'amoatrac«i^taatadmath«natksalavalla: (Martc ona) 

0 Gradt 2 or towar 0 Grada 7 - 9 

0 Gradt 3-6 0 ^0 ^ 

What in«nictlooaladaptatk«a%w»iWtWattudamnaadlnpoa«aacon^ 

0 ParaHal modifiad curriculum 0 Tapad »xttJOOks ® Inwrpratar 

0Tutof3 ©Untmiadtafis 0 Otfw (Spaofy)- 

0Notatakars 0 Taat gwan orafly ©Nona 

©Computara 0 Taatiogoutsida group 0 Postaacondary not appropnata 

0 Eniargad pnnt ® '"^ 



H. Problpni Sphaviors 

1. I>Paatl*«ud««a»li«ifcanyp««««b.^ 

•nttrtara aariauite wMi a ii ai i uyfc«ty vtd fpeial a#iatmanla. or fffglgt titnTOtfffllt fMidaiteai ano training 

0 Yaa 0 Ma 0 Oon't know 
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if yaa. marH al tfiat applr- 

0 ExNbitiN^unu«ialbalMMor 

(e.g., ech o i ng , dicfcmg. rocking} 

0 Droolt 

0 Touchaa othars inappropnataly <a.g„ too huggy) 
0 \s vioiant or agt^aawa towarda ottwri 
0 Exhibits inappropnata $axual bahavwr 

(e.g., masturtMtion, axposura, soKotation) 
0 Hurts satf 
0 f^ns away 

© Dafacas or daatroys proparty 

0 Shows cnminal b^avior {a.g., staats) 

Q Has baan involvad with cnminaJ justica systwii 



0 tem^ 

0 Withdraws, rafusaa to spaak 
0 Shows sevara dapratsion 
0 Practicaa subatsnM abusa with dnjga 
0 P i acticaa Mbatvica sIkm with alcohol 
0 Exl^faitt hyparactivity 
0 Othar (Spadfy:) 
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Pteasa racofd stop tifn« 
hm — ■ — ► 



m MINI 

001®^ 
G0OO 

I 0i®0 
I 0®0l 
0001 
0001 
0 0 
0 O 
0 0 
3 0 




AMVOCANNitllTUTES 



PAtOAUOOmCf 



ITU 

MsMBlCAMOS 




Performance Assessment for Self-Sitf f idency 

(PASS) 

Instructions for Raters 



In this field test of the PASS instrument, you will be asked to specify: 
. descriptive student information, including major problem behaviors (PASS pa^ 1, 6. 7) 
. ratings of student performance, including settings for the rated performance (PASS pages 2-5) 

To do this, you wiU need: 

• a No. 2 pencil or a blue/Wack pen 

• a watch or a clock you can see 



REMEMBER: 

. Please mark your answers clearly in pencil or blue/black ink by completely filling in the 

appropriate bubbles and spaces. 
. Look at your watch (or clock) and record the time in the spaces provided in the instrument 

(top of PASS page 2, bottom of page 7). 
. Review the entire form after you have compIet«nt to be su« you ^^^^ reIpONSES 

FOR EACH ITEM. Be sure to ERASE ANY ERRORS OR UNINTENDED RESPONSES 

COMPLETELY 
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special Instructions for Completing the 4-page Rating Scale (PASS pages 2-5) 

Each pag« erf the 4-page scale lists behaviors required in a particular area of regular adult 
life whether or not a person has a disability. You are to select the aroropriate rattng code 
based on what you have observed, or what you know about the student from other 
knowledgeable persons. 



fefmniaiKc Ratii^ Codf* 

0. Unable to rate 

1. Does not or cannot do 

2. Does or can do with extensive assistance or supervision 

3. Does or can do with some assistance or supervision 

4. Does or can do independently 



EXAMPLE 

0 1 2 3 < c. Communicates basic needs 

IT I 

The selection of the rating code "3" for this sample item indicates that the rater feeb 
the student needs some assistance or supcrvison in order to communKrate his or her 
basic needs. See page 4 of these Instructions. 

EXANinE 

0 2 3 4 h. Takes part in leisure-time social activities or organized sports 

The selection of the rating code "0" for this sample item indicates that the rater feels 
unable to rate this behavior with confidence or accuracy, given what he or she knows 
about the student. See page 4 of these InstnKticms. 

EXAMPLE ^ , ^ 

0 12 3 4 



S W K O 
• ® ® ® 



^ T (D ^ ^' follows rules 



® (D CD (D ® ^- Follows through on plans, decisions 

You are also to indicate the settii^s) in which you know that behaviors are 
performed at the Wveb you have rated. The selection of the rating code S for this 
^mple item indicates that the rater knows about these student behaviors in the scliool 
setting, but not in the other settings. See page 4 of these Instnictions. 

EXAMPLE 0 12 3 4 

"" ^ (T -2^ (3) ® ^ Identifies values of coins and bills 

S W H O 
• • © ©"^ 

^ ^ 1" (J (3) e. Calculates checkbook balance 

The selection of two rating codes, S and W, indicates that the rater .^^'y^^;^ ^ 
knows about) these student behaviors InTbth the school and work settings. See page 4 
of these Instructions. 
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Service Cathodes and Definitions to Be Refined in Project PASS 
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Peggie Can^ieau, Prqject Dixector 
American Institutes for Research 
Palo Alto Office 
P.O. Box 1113 
Palo Alto, CA 94302 

NOTE: lUi^toan^^UM^ier^a^fn^^i^^ 
ad^tUms/nbstitutioHsJ. 

MOBIIJIYSERVICiS(<^tl«ns^»eciaUiedtnu^ [^^'S??*,*^^ 
TraininE related to pesonal nKMity ft» inilividi^ 
tacSi desttoitfim tnintog, map 

mmsportaion^wmadmoncf specific exmn^ 
Tramportasion Services.) 

SPECUUZiai TRANSPORTATION SERVICiS: Acqa^ticm of gjedalized 

oan^witaticHi, including q»ecial public tmn^KHtaticm, ^^J^^^^y^^:^^^' 
ooZunity Uving, or receipt of other sendces^ CAcquisitim f oM^ w cU^ thm the 
category includes pwvision cf transportation, not training mO^use hereof.) 

TECHNOLOGICAL AIDS AND ADAFITVE EQUi™^ Evaluation for 

and piovision of equipment, midlines, prostheses, aids, and adai«ive <^(»s necessary^ 
cmploynwnt, education, community Uving. or lecdpt of othw services, mdudmg eqmpment 
and devices for communicaticm aiKl mobility, (no changes) 

MEDICAL AND MEDICALLY-RELATEa) SERVICES: Surgoy, medication physical or 
occupational therapy, and (Hto medical treatn^ts or ser«c»s related to the (no 

changes) 

COMMUNICATION SERVICES: Assistance and training tdated to the act of 
communication, including ^«ditoguage thenipy, interprets serr^. rc^ 
braille training, and tactile intopieting services. ('Speechnanguage ^rapy added to 

expand examples.) 

INDEPENDENT LIVING SERVICES: Training or assistance in performing baac daily 
Uving activities (e.g., cooking, laundry, hygiene, [transportation,] tii»c man^ancnt, 
financial management) to enable iiuUviduals with disabiUties to Uve w^h mMtamm 
toSSS^coi^ CIYansportation deleted to maintain inugrity 
Tra^rtation and Mobility services; ending phrase modified to em^iu independence as 
end goal.) 

OVER 
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RES]j)ENT]AL LIVING SERVICES: Pn^iams to itoe or mamtam individual with 
4isainlitics m re^teotiai fiKilitks (e.g., gnnip hcnnes) isr in stn^^tuicd community livii^ 
anangements (eg., aqicrviwl apaftn«n»). [indiMling daily living skills training and related 
services] (ServUx imiuda ^xment as ^fdl as maimGumc^^ 
dw!^ to eUminM averk^ wiA bid^^sndent UvUig ServUxs,) 

SOCIAL SKILLS TRAINING SERVICES: Trainii^ in the (tevdopnwnt of intwpcrsaial 
sodal skills aiKl social con^jetendcs essential to wnjdoynwit, education, and community 
living, (nodianges) 

MENTAL HEALTH SERVICES^ Peramal adjustiMt amsding, sex ediKation or 
counsding, psychiatric or psydwk^ical therapy, subaancc abuse programs, ami mratal 
hcalfe services, (nc changes) 

VOCATIONAL TRAINING AND JOB PLACEMENT SERVICES: Vocational 
assesamnt and time-Umited services that imdiKie simile job rderral and jdacttiont, 
vocational education programs, on-site training and plM«nent, ami shwt-tenn finandal 
assistamx. (no diaries) 

ONGOING EMPLOYMENT-RELATED SERVICiS: Vocational assessment and long- 
tam or poiodk services Mces^ to maintain individuaU with disabilities in suitable 
enqrioymeat, including training, job cowdies, aiRXOted wnjdoynwnt jaograms, shdtered 
cm^c^roeot programs, and other siqjport s«viccs rdaled to oi^ri^^ (nochanges) 

ALTERNATIVE iDUCATION SERVICES: Programs ft» continuing aiult education, 
including Adult Basic Education (ABE), Gewral Edwational Devdopn«nt (GED), adult high 
school diploma, and adult compensatory or ^jecial education, (nodumges) 

SERVICES TO SUPPORT POSTSECONDARY EDUCATION: Special s«viccs 
iMoessary to enroll individuals with disabilities in technical sdiod, community college, 
college, or university, and to maintain individuals in those in^tuticms to readi ^wafied 
[d^ree] tevd of cetllkatioii; indiaSii^ counsding, curriculum adaptations, aiMl academic 
suimwt. [note-taking services] ('kvel cf cert^Uatim' us&i w broaden d^iMon of 
cmpetency beyond (mfy a degree; 'academic smtort' sidfstimt&i to pro)4de a more general 

example.) 

RECREATION AND LEISURE SERVICES: Training ami suRJort to «iable individuals 
with disabiUtics to engage in apfm^niatc recreatiOT and leis^ (nochanges) 

FAMILY SERVICES: Suppon or assistance, including counseling, to femilies of 
imiividuals with disabiUties necessary for the rehabiUtaticm or increased indepemteice of the 
imiividual widi a disability, (nodianges) 

CASE MANAGEMENT SERVICES: Referral, evaluation, and counseling neected to I^p 
persons with disabiUties Mcure sevices; including coordinaticm, follow-up, and advcjcacy to 
assure delivery of aqjpropriatc sendees, (no changes) 



o 

ERIC 



23 
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iNsmucnoNs for round i ofthe oELPia raocs^ 

Fo, fi* ^ ^ eategnriei, vou will tdastitf UP <o tfw cficat cbtractefatici thit ire Ifltgiy to 
trteadedUcwthattfwiefvkeiicssfRija/. White the* »• m«V ^ifawl chwrtoirtta 
qniM f^t—i tha MMd &» M partieiilir ieivice. we tie toofciaK far the pmm k wih ope» tfa«t wpnid 
trteryoordfiditaL Sm Cobann A in Om Exan^ bdom. (If there •» more titan toe cfieitt 
cfaafBclBriiticittoifaBWOiddrankas/«m<iiiftf,w^ Its NOT 

ngem^ to rmUe-of^ ^ma dimusoi^ that yw idm^ 

We wffl awtme that tbe key dunctemtki ycm 

(&^)ffity(ies). Cofttmn A in <te £&srr^ 

Yob mey glw ^»erk dumaeristke (64, mo^ 
fyM t^ tftead CO own ti^lrtii^ wecb). 

If yoo do DOC fed qua&fied to respond for • pMticutar service cttegofy. mark that page i>on lr A>mw 
It tbe top of the fono. 

If in your opiaioB the d^nWon of a partkalar service ctfegoiy 
direafy on At fom. 
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CLIENT CHASlACTBRISTKSa»tu»4o«l ttutaOkmL tfcJ 
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TYPE (i) OF DSARUTY (isi) 
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SERVICES TO SUPPORT POSTSKONOARY EDUCATION 
II m ri mt H m lni1i " ti rr " — ^—^-^ — «- 



CUBNTCHARACIERlSllCSfMttTtora «t»»iwto»l. tit) 
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iMtniGtioBt for Rimnd 2 of tbe Ddl|M fism^ 



For eadi of 16 seivke categnks, you have been provKkd with a summaiy of the dtent 
characteristics that field eiperts ia Rouiwi 1 indicated were Uke^r to trig^ their «^on 
that a particular kiwi of sendee was esscntiaL Eadi summaiy indudes an outlii» of the 
major Sent charactcristia ktemificd by the eiperts (mifklle aHumn), as weU as swm 
mdfic examples of ttoe dient diaracteristics as ciwountered by the eiperts (diadcd, 
Sit-hand column). The aitlines attempt to capture the infonnaticm pfOvi<ted by cspcrts 
Mdeariy and comprehend as posable. (Note that the outlines are not intewled to be 
consistently parallel in content or grammatical constracticHL) TTic mind»r m^ntbescs 
besicte eadJ lettered section of the witline is the number of experts who iitentfficd 
diaracteristics under tl»t letter beading. The number in parentheses beside eadi Roman 
number heading is the aggregate number of diaracteristics proviited by experts withm the 



entire set 



A orimary purpos.^ of Round 2 is to identify those areas of dient diaracteri«ia that 
should be iwdudcd in the dedskm mks for tl» PASS Expen System, will convert 
infbnnation about dients into estimates of antidpated service ne«b. (The d«3«m niles 
<»mimse the "If-then- rule structure that wiU be used to pro^ senr^ 
thcwpert system must define a finite and economical set of rules, your task is to identify 
onlydh^ dient diaracteristics that are assodated with an essential need Imt the service. 

A second jHirposc of Round 2 is to assess the adequacy of sendee list definitions, prior to 
developing tlte PASS Expert System. 

Your input is essential to acain^lish the above goals. Please foUow these steps in 
completing Rou^ 2 of the Delphi Exercise, for each scmce cate^ry: 

1 Indicate, wtth a aeck maik, whether each dient diaracteristic presented in the 
middle column-''.4«» ofSen^ Need Identified by Field in Ramd l'- 
should be induded in the decision rules for the PASS Expert System: 

. Check "Vefy In^oitaal" if that dient characterisdc shcraM he i^l^d in the 
dec^km rutes-that is, if that characteristic would dcfinitefy trigger your 
dedskm that the service was essential. 

Oie^ •Uataportant* if that dient characteristic should NOT be iKladed in 

the d^i^on rules. 

Check "Uadedded" if you cannot dedde whether or not that dient 
diaracteristic should be induded in the decision rules. 

Be sue !ti pot one check mark best4k eveiy section of the ootUne with a 
Roman nnmeral and letter heading. 



Alio Mli tfcjtf 7^1 ^ NOT Bccd to auUee any tedi^mi refirdl^ Of qpedfle 
^ipiiia im tke skaded, rlgbt-hand colnma. The examp^ uv |ms¥ided for 
lUmtratlvi pvipotes oaty. 

Towifd the end of scm service categorica, you arc asked to detcnniM whether 
selected dkm characteristks should be nuived to another service cat^^ In these 
cases, imiicatc with a check mark: (a) whether to include the diarav^ristic in the 
Expert System ctedsion roles, as you did in Step 1, and (b) whether tl« 
^ncteristic shaiW in feet be moved to the specified setvice category (or to 
another categoiy you feel would make more sense.) 

If you do not feel qualified to re^nd for a particular service category, mark that 
set Don't Know at the t<^ erf the fcm. 

Hease indicate Bay commoti ycm have about the service area definitions, the dient 
diaracteristics or example, the organization of the categwiesj w tlM 
^HmHmateness of their indvsion in the ^>aoc provided. Feel free to edit (change, 
cross out, or add) or comment directly on ibe form, as ym wish. 

At tl« end of the Round 2 packet is a chart (color^ page) entiUed "Add^oiud 

SerweOtfMorttt" AfewejqjertsinRou^ IsuggMtedthatsev^ 

categories be aiWed to the 16 already (tefiaed Please iaJicate on tl» chwt ^ther 

each cateiwy should be dassified under an emting service category, or whether 

that cate^ sh<Hild noC be included within the PASS Expert System fran^rk at 

all 
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MBNTAL HEALTO SiKVIXS . 
Uhen^y, 8ub8taf]c& si)UK pn^nim and oUier menUi heaHh aer?k3e& 



or nrariMi 

for liKluaoo in Ibe 
Project PASS Kipeft Sjitem 

Very Unim- 



Itxtm s>( Senke Need 
UmUneot 



(Sefecied Sampka from IkM Espnta) 
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k ShD«i««nio(depitHioQ (14.25,26.28. 05. 
10. 27. 33. 04,01. 12. OR) 



a EsiiibiU iStgef/HP^'^ bebftTior (04, 25. 
29. 33. 08. 16, 23. 24. 32. 36. 05. 26. 20) 



01,26, 33. 10, 22, 28, 09) 



kiMikUW(e«. ot^MmiM. Ufa o( 

Molkf to Ife) 
k nitl«l(«im id im^mDg to p«ctfc%^ In cooununity 
UIm 1 hWi/wiinnwn^i *ae to iepinriDo 

TkumX ml 
UomX ikep 

Butm/ihtmUm to hKm Hlf or othets 
Hm kiBpproiMMs ootbomU to imbiUty 

fa fftthdmmi/ou^ of tottsh ith mMf 

maixm ffg^bMs^ybKikx, ^^^^^ 
oiitimk ilan* Iwi p^jcboUc bred* ktm$^ 

faaet AtUs^ tMfe aee^ i&. is 
unkBmpt, h^«d|fat, bis no liWrE* 
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for lttM»kithe 
fVo>ci PISS Eipcsi Sfdeax 
(Check (ME) 

Very Unim- 
kDsertAot portant UidecUM 



lyiMmi PUPtefai tmtiiMai 

n ftt BiBd pnbkoH (IC. 10. 20. 2&. 26. 19. 22 
24. 28. 31. 13. 12) 



pliylapiil*) 
Ebei not pnctin vie nx 

BipnM Mra for IBS tiooAUDa 

(^DDoi Goolnil sniii «!■ m i<>t> pub^ 



1. ^ tAher BjTOptooM/iiffM Indiottln^ 
iimtmeal (16) 

1 ExMbiU mkioML behn^ oq the job 
^S, 13. 30.31) 



E Rspran emsA for M^fflnrr? 128. 32) 
C ^ fimt diffii^y M»e{iiia|/*jji^^ 



Uwbfo to Imp {MtDfMl pn>bliBDii (Hit o( 

flPfirtunMnt' 
b t»ik« tnt^ with menobore o( oppoiKe ici 

at work 

Ualifo U» get tkoi «tth o»-i«kn 



tmi. msmM ptkA 
BkM tioubfe th^iiDg iokmsi wffmim fassdkomi 



a Cknnoi control behavior ia idsDoi 

IL (kaoot accept n«po«bibiUly for oira 
foOum/tdames othen (Id) 
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eOFUfORTANCE 
(or fDcfavioa la the 
Pit>^ PASS EiiMri Sjiiem 
(diackOfC) 

VOJ VCDkCDr- 



Aevm o( Serrke Neml 
iMiflbi bj fUi Eiiwdi ia Rbiu^ 1 

am of wfaiUoee tbum (fi. 10. U. iS 
26, SO, IS. U) 

k kfgmt to hAie bem under lofiueoce oi 
•kohol/dhwi (04. 27. 2S) 



E Bm other rifoi o€ •kehoi/irug prDbkmi 
(t7. OS. 36. 17. 20) 



IV. Shom iifDS btAof ^suKd ph|«iaUj? or KxuaUy 
i EI^N}rtMtotMTeb^ftbtMdi(17.32) 

a E nEi— eoacrnn inbout gmal. |ihjpkal, or 
■ubttuee tbun (4) 



£flkulL7 ki eootikaJikia/tftesh 
Hu Mkurai from mManft sInik 
Fro^ucUvtty reducei iue to ftkohoi tbuK 

Hm kMi icMti U> ticoltti 

iiR umlLei under the kiAuenoe 

fa fresQuec^ 'licb" from imt iue to Akohoi proNem 



mMmf tm (VVmrn vfi0Bm» ymvF sfimmmiM to U» outJicM by number or teitar. Cbotioiie ooaufientj oa hmek of jpife 
U necemuj.) 
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tenk» €»Letxf. Shouli the iUm be lOVffi to ioother cilW^ (hesk oi wMe jax vmoom kx \be tgtem 



for kvkitebUK 
Pko jed PASS Eipiii S|iUm 



(Check ore) 
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Infomtkm and lortriKtioiis for Round 3 of tbc Ddphi Emdse 



Purpose 

The primaiy purpose <rf Round 3 of the Delphi U to "map" dicnt diaractcristics th^ 
experts cwisidercd esMtlal for inchision in the knowiwlgc base for tl» Prefect 
imrt System <mto tl» content areas in the PASS instrument anl to ictentify gaps mthe 
instrument's coverage. (The project abstract we maflcd you a few numths ago desoibcd 
this draft instrument, whidi is stiU under (tevclcnmient awl wiU be refiwd on the basis of 
the DehAi process and a national field test) Y<wr input frcmi Rouwl 3 wiU be used to 
develop a preliminary set of decision rules by whia the PASS Eaqjert System wOl link 
data obtained from the PASS instrument to estimate of anticipated service needs. Your 

comments wiU also be used to refine the PASS instrument prior to the field test. 



VVhy the lists Are Shorter This Time 

For each of 16 service categories, you have been provided with a shortened list of the 
cUent characteristics ip^-hmd column ofmA page) that expert consensus m Rouiui 2 
indicated were V«y Important to include in tlw knowledge base and rale structure for 
the PASS Eaqjert System. In our analysis of responses to Round 2, we established a 
threshold value, or «<io, by which to identiftr those character!^ 
conscnais indicated were essential to inchide. The number in parentheses beside each 
characteristic is the rating index for that characteristic Tlw calculation for the index was 
made as foUows: Response were coded on a three-point scale (+1 « Important; -1 « 
Uninmortant; 0 - Unctedited); sum valu^ were calculated for eads dient diaractcnstoc 
and were then adjusted by the total number of rcspomtents wIk) rated the diaracteristic 
to (*tam a prmwrtional rating, or "rating index" (le.. tl» ami value was ^vid^ by the 
number of reaxmdents to obtain an adjusted sum); a threshold vahie of .60 aiMl above 
was established to determiiw which characteristics to consider mdudmg m the expert 
system kno«^ed^ \mt and rule structure. 

Each list also reflects your si^^tions for refining the wording and for moving selected 
items to a diffisfwit service category. (We are still analyzing your suggestions for 
improving the definitions of service categories.) 
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In additkm to the 16 li«s. you have also been provided with a CraMt OitfUae fiN- tbe 
7ASSAueammlaMtrmeai(bbiesh^). In this 'nui{q>tn^ exerdse, yew will ii^iicate 
oiM m m«€ tqric areas cm the PASS Content Outline with whidi each characteristk in 
the lists m^t be assodated. As a convenience, we have already indicated in the fc/ir- 
fuBid cdumn <rf eadj page wtere we wcmld jrface tl» characteristics in the ri^t-hand 
cdumn. D^iKlii« on the qiedfidty of tl» diaractcristic, we have in some ca^^ 
saiggcsted a main ti^c area (e^ A, BX a group of t«^c areas (e^.^ 
Atopic area(s) (e^ A-1. M). In stfll other cases, we have ^^J^O CODE) to 
indicate that the djaracteristic docs not, in fact, s^ to fit within tl» existing content 
areas on the PASS Content OutliiK. (This signals a possible W in the coverage 
present^ provided by the PASS instrument) 

Please foUow these steps in co^^)lcting Round 3 of the Delphi Exerdsc, for each service 
category: 

1 Read the entire list of diaracteristics in each service category to get a sense of how 
that auegoty fits into tl» PASS Content Outline before attempting to make your 
decision about an indhddual characteristic. 

2. For each characteristic. iiKlicate whether you AGREE or DISAGREE with the 
topic area(s) we have assigned-that is: 

If you AGREE with the suggested topic area, put an "X" or check mark in the 
"Agree* column. 

If you DISAGREE with the suggested topic area and believe that another 
topic arca(s) is more ^>prq>riate, spedfy the vc^mtd topte area(s) in the 
TMsagree* column. 

. If you AGREE with the suggested tc^c area but also want to ADD a topic 
area(s), put an •X» or «eck mark te Ihe ■Agree* cohnm AND speciQr tlw 
addttkmal toj^ am(i) in the "Disagree" <»lumn. 

Be sure to mark the "Agree" and/or "Disagree" column beside EVERY 
characteriidc tiiat has lines beside it Sec the EXAMPLES on the Mowing pages. 

If you <to art feel qualified to respond for a particular servi(» category, mark that 
set "Don't Know" at the top of the form. 

4 Write in the Corameits box if you have further aiggestions abwt the Content 
Outline for the PASS instrument, the dicnt diaracteristics, or the service area 
definitions. Also feel free to edit or comment directly on the form, as you wish. 



3. 



EXAMPLES? 



MpgUTY gKVCES (oto tim specialized tja n s p a t atkn aa-wsa) ^u^... 
ftmif itfl r^, couBadinfi tr trainiog reSated to pecsnai mflbility iae individuals with dsabiUlies 



on CboUot OuUiotf 
i-S 



i-6 



t 



L ftf mv ptipkal dlMbiBtf UmI nqpinsi the tiM 
ol i iyir''**^ if[^piD«tt for tmbokiioa (4}3) 



£rpeff £^e5 f/ixi/ f/iesc chamcteristia belong in sub-iopic ana */-5, 
Mobility and Trampcnation SldUs. " 



CASE MANAGOCENT SK?CES 

!te(eini evtiuatiDD, and axmseiiDg oe^sded beip 

induiing cofxdimtta fiaOoir-uii, and advocacy to 



liUi dtahaitiwi aecure services; 
delh«y of approfiiiate 9ervice& 



SuoeiUd Tbpk iMi) 
oa GDotcot OatliM 
(blue iM) 

F 
f 



no 



Anm of Srvkf Niii ItoiifM "Vicy bnixxtani'' 



A. Stm diMiaty itafite oofolof 



// ///^; Expen feels tkm characteristia do not fit cieoFiy into emtmg categories 
and assigns "F, No Code." . . 

IV/IVA: Expert agmes thai "F' is ap pro pnate' because chamaemtics appty to 

fgpiih^ not client 
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MBfTAL HEAUH SERVESS 



SufiBitaii !bf)k Ana(i) 



Qanani fi( fxparti ia {buad 2 



8 

E 



4 ^ Appnrs b hAiv hm iBMte laniMoct of 

ftlQob)!/^ (^9) 

III/III^: Expert agrees thm substance abuse belongs in % Major Pmbkm 
Behaviors,'* bui feels U also, belongs in "A-Z Healik and Safety." 




liaiBini and awppat to BMbte in^^ 



oaCbotiQi OdlliiM 
Hblmskmt) 



B-4 



B-2 



IIIA/D: Expai agrees with sub-topic areas^ including iisili sub-topic areas for 
IILD, (Note thm main heading III does not require an assessmem 
sines it has no Ones beside it ) 
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